This disclosure relates to computer vision on an unmanned autonomous vehicle (UAV) and more particularly to determining camera pose rotation based on the attitude of the UAV for computer vision.
There is much excitement and hype, surrounding technology for unmanned aerial vehicles (UAV), commonly known as UAVs. Despite ample progress, several substantial technical challenges yet remain before “ubiquitous UAVs” becomes a realistic vision of the future. One core, well-established challenge is in enabling UAVs to engage seamlessly with their environment. Addressing this challenge seems to demand ample computer vision, and in real time.
We are considering the need for computer vision on small flying UAVs/unmanned autonomous vehicles (UAVs). We believe computer vision will become increasingly essential for engaging with humans, for operating in unknown environments, for spotting targets to find or follow from a distance, for comprehending and responding to the visual or multispectral data they collect in real time, for collaborating with other UAVs or robots, and for finding means to recharge or refuel. Given these visual needs, capabilities for real time computer vision on UAV platforms seem like a must. Further, as these functions may often be safety critical, it may not be wise to offload to powerful back-end cloud compute systems. Much like a self-driving car, UAVs must also be mostly self-sufficient in their compute needs. However, unlike a self-driving car, the constraints in form factor, weight, and energy in the compute platform are highly constrained. Flying with a small low-power system on a chip (SoC) like the Raspberry Pi is quite feasible. However, a high performance server with GPUs is not currently realistic. Indeed, the resource constraints endemic to mobile computing on smartphones again apply to mobile computing on UAVs.
However, computer vision heuristics are often heavy weight, and difficult to run in a computing platform that can fit in the weight and form factor limitations of most UAVs. Several prior works consider means for precise relative GPS tracking; GPS applied to “attitude” (orientation) estimation, especially in the context of airplanes, satellites, and ships; and the use of GPS “position” in computer vision. However, these techniques are very time consuming and therefore are feasible for obtaining real time data.